Ultrasound image enhancement and speckle mitigation method

ABSTRACT

A method for enhancing an ultrasound image is provided, wherein the ultrasound image is segmented into a feature region and a non-feature region, while sufficiently utilizing features contained in the ultrasound image, in particular including some inconspicuous features, the enhanced image according to present invention is not susceptive of the image segmentation and avoid dependence of the enhancement effect on the segmentation template, so as not to produce an evident artificial boundary between the feature region and the non-feature region but to highlight some special information in the image and to remove or mitigate invalid information. Thus the enhanced ultrasound image is particularly suitable for the visual system of the human beings.

FIELD OF THE INVENTION

The present invention relates to ultrasound techniques, and moreparticularly, to data processing techniques in ultrasound imaging,especially to an image enhancement and speckle mitigation method forimproving the quality of ultrasound images.

BACKGROUND OF THE INVENTION

FIG. 1 shows a block diagram of a typical ultrasound imaging system. Itoperates as follows: under the control of a main controller, a probetransmits an ultrasound to a body tissue to be tested, and receives anecho signal reflected from the body tissue after a certain delay. Theecho signal is fed into a beam former, which performs focus for delay,weighting and channel accumulation to generate signals on one or morescan lines. A detector detects the scanning signals output from the beamformer and feeds them into a DSC (Digital Scanning Converter), where acoordinate conversion is implemented. The resulted image data is sent toa computing device (generally including a built-in processor, a FPGAcircuit, even a computer system or the like). An image enhancementmodule located in the computing device processes the image data andfeeds it into a monitor for display. Alternatively, the ultrasoundimaging system may invoke an image sequence stored in an externalmemory, process them by means of the image enhancement module and thendisplay them.

Here, the ultrasound imaging system employs the image enhancement moduleto realize post-processing of image, which aims at improving the qualityof ultrasound images and assisting medical diagnosis, in particular, byovercoming problems in two aspects. The first aspect is to enhancesignificant structures or features in the image which interest thedoctors, including bones, capsules, canals, cavities or the like. Thatis to say, all distinguishable structures should remain in the resultedimage after ultrasound image enhancement, including normal and abnormalstructures, while providing sufficient textural and contrastinformation. In the second aspect, speckles should be suppressed. If thereflection surface of a tissue within the human body is not so smooththat the coarseness of surfaces are equal to the wavelength of theincoming ultrasound, the echo signals generated by different reflectionsources may overlay or counteract due to phase difference. Such anoverlay or counteraction is represented visually as grains of the image.As such, speckle noises are always present when scan line data atdifferent locations are processed and combined to form a finalultrasound image. The speckle noises will mask some useful informationin the image, thus interfering with the doctor's diagnosis to someextent. Speckle mitigation is thus another object for enhancing image.

In relation to the above problems, an image enhancement method with agradient-based segmentation is disclosed in U.S. Pat. Nos. 6,208,763 and6,592,523. In this method, an image is segmented into a structural and anon-structural region according to the gradient information. Anisotropysharpening is performed on the structural region based onIntensity-weighted 2^(nd) order directional derivative, to enhancecontrast of edges of the image. Isotropic smoothing is performed on theimage data classified as a non-structural region, to mitigate specklenoise. The above prior art, however, has the following disadvantages:(1) although the method in the patent advantageously segments an imageinto a structural and a non-structural region, but it leaves out someinconspicuous features by only taking gradient scale information intoaccount, in particular, the enhancement effect relies too much on thesegmentation template when the structural region takes the gray-scale ofthe image as the sharpening coefficient, and too much discontinuity willoccur at the edges of the segmentation template. Accordingly theenhancement effect is not ideal; and (2) linear smoothing of thenon-structural region cannot mitigate speckle noises greatly.

SUMMARY OF THE INVENTION

The present invention is made in view of the above disadvantages.

According to on object of present invention, there's provided a methodfor enhancing an ultrasound image, wherein the ultrasound image issegmented into a feature region and a non-feature region, whilesufficiently utilizing features contained in the ultrasound image, inparticular including some inconspicuous features, for enhancing theultrasound image.

According to another object of present invention, there's provided a newmethod of processing the feature region, which can make the enhancedimage not susceptive of the image segmentation and avoid dependence ofthe enhancement effect on the segmentation template, so as not toproduce an evident artificial boundary between the feature region andthe non-feature region but to highlight some special information in theimage and to remove or mitigate invalid information.

According to another object of present invention, there's provided amethod of mitigating speckle noises, in particular black speckles in theultrasound image.

A final ultrasound image resulted from the method for enhancing theultrasonic image and the method for mitigating speckles is particularlysuitable for the visual system of the human beings.

According to one aspect of present invention, there's provided a methodfor an enhancing ultrasound image for use in an ultrasound imagingsystem, comprising steps of:

A) reading the ultrasound image;

B) segmenting the ultrasound image into a feature region and anon-feature region based on gradient information and gray-scaleinformation in the image, and then performing different data processingon the image classified as the feature region and the non-feature regionrespectively;

C) merging the processed feature region and non-feature region toproduce an enhanced image.

Preferably, wherein the step of segmenting the image comprises a step ofextracting distinct boundaries in the image by means of the gradientinformation and a step of extracting brighter region in the image bymeans of the gray-scale information to generate a gray image template.

Preferably, wherein the step of segmenting the image further comprises astep of extracting weak boundaries in the image by means of varianceinformation in the ultrasonic image, to generate a variance imagetemplate.

Preferably, the step of segmenting the image further comprises a step ofpost-processing the variance image template to produce a segmentationtemplate, wherein the feature region being set as the image regionindicated by the segmentation template.

Preferably, wherein the step of post-processing the variance imagetemplate further comprises a step of averaging the segmentation templateof continuous frames, wherein for a feature region newly added to theimage template of current frame, the feature region is filled in as apart of the segmentation template only if several continuous frames forthe feature region satisfy a segmentation condition.

According to a second aspect of present invention, the data processingon image data classified as the feature region comprising a step ofanisotropy smoothing the image data which comprises sub-steps of:

-   -   a) determining a local dominant orientation for each pixel; and    -   b) smoothing each pixel in a direction normal to the local        dominant orientation.

In a preferable mode, the data processing on the image data classifiedas the feature region further comprises a step of anisotropy sharpeningthe anisotropically smoothed pixels, comprising sub-steps of:

-   -   a) computing an orientational Laplacian for each of the smoothed        pixels in the local dominant orientation; and    -   b) multiplying the orientational Laplacian with an        image-sharpening coefficient R_(sharpen), to produce an        anisotropically sharpened result.

In a preferable mode, the image sharpening coefficientR _(sharpen) =C*std*(M _(frame) _(—) _(ave) −MID)/5

Where C is a predetermined constant, std is the standard variation forgray image within a neighborhood of each pixel, std=√{square root over(E[I−E(I)]²)}, wherein I represents a gray scale intensity,

M_(frame) _(—) _(ave) is the template frame-averaging matrix used whensegmenting the image, and MID is a predetermined value corresponding tothe template frame-averaging matrix.

According to a third aspect of present invention, speckle noise issuppressed by locally adjusting the gray scale intensity of the imagedata classified as the non-feature region, comprising

a) dividing the input image into blocks, and computing a gray scale meanof each blocked image, Mean_(block);

b) setting different gray-scale adjustment coefficients R_(adjust),based on the different gradient values of various pixels in the inputultrasonic image; and

c) computing an adjusted gray scale intensity I_(adjusted) to produce alocal gray-scale adjusted image,

I_(adjusted)=Mean_(block)+(I_(in)−Mean_(block))*R_(adjust), whereinI_(in) represents the gray scale intensity of the input image.

In a preferable mode, black speckles are further suppressed byprocessing the local gray-scale adjusted image, comprising:

i) computing a mean image represented by its gray scale intensityI_(mean) for the input image or the local gray-scale adjusted imageI_(adjusted) by using a neighbor domain of each pixel with variablesize;

ii) determining a compensating pixel and its respective compensatingamount based on the mean image I_(mean); and

iii) adding the compensating amount for each compensating pixel into theinput image I_(in) or the local gray-scale adjusted image I_(adjusted),to remove the speckles from the non-feature region.

In a preferable mode, the compensating pixel means a pixel for which animage difference I_(diff), obtained by subtracting the mean imageI_(mean) from the input image I_(in) or the local gray-scale adjustedimage I_(adjusted) is greater than a predetermined value; and thecompensating amount for the compensating pixel is multiples of therespective difference I_(diff).

With the above technical solutions, the feature information in theultrasonic image may be extracted and enhanced more effectively, and theimage enhancement effect relies slightly upon the image segmentation,while it is advantageous to mitigate black speckles. The enhanced imageis thus more suitable for the visual system of the human beings and morehelpful for reference in medical diagnosis. The above and otheradvantages of present invention will be described with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an ultrasound imaging system accordingto present invention;

FIG. 2 shows a diagram of enhancing the ultrasound image according topresent invention;

FIG. 3 shows a diagram of segmenting the image according to presentinvention;

FIG. 4 shows a flow chart of generating a gradient image template inimage segmentation process according to present invention;

FIG. 5 shows a diagram of generating a gray image template in the imagesegmentation process according to present invention;

FIG. 6 shows a flow chart of generating a variance image template in theimage segmentation process according to present invention;

FIG. 7 shows a diagram of post-processing the variance image template inthe image segmentation process according to present invention;

FIG. 8 shows a diagram of a template frame-averaging process accordingto present invention;

FIG. 9 shows a diagram of anisotropy smoothing the image data classifiedas the feature region according to present invention;

FIG. 10 shows an orientation in variance computation according topresent invention;

FIG. 11 shows a diagram of anisotropy sharpening the image dataclassified as the feature region according to present invention;

FIG. 12 shows a diagram of merging the feature region according topresent invention;

FIG. 13 shows a diagram of the local gray-scale adjustment process for anon-feature region according to present invention;

FIG. 14 shows a comparison between the histograms before and after thegray-scale adjustment of the non-feature region according to presentinvention;

FIG. 15 shows a diagram of mitigating speckles in the non-feature regionaccording to present invention;

FIG. 16 shows a diagram of detecting black speckles according to presentinvention; and

FIG. 17A and FIG. 17B show a diagram of the data processing on the imagedata classified as the feature region and non-feature region accordingto present invention, respectively.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Detailed descriptions will be made below to the invention, inconjunction with a preferred embodiment as shown in the accompanyingdrawings.

The invention can be implemented with the ultrasound imaging systemshown in FIG. 1.

The method for enhancing an ultrasound image as provided in theinvention is used by an ultrasound imaging system to optimize display ofan ultrasound scanned image. As shown in FIG. 2, the system first readsthe input ultrasound image data and then segments the image into afeature region and a non-feature region according to the gradientinformation and gray information in the image; sequentially, performingdata processing (1) on the image data classified as the feature regionand data processing (2) on the image data classified as the non-featureregion, respectively. At last, the processed feature region andnon-feature region are merged, to produce enhanced image datacorresponding to the original ultrasound image, and then the enhancedimage data are output for display or storage. Detailed description willbe given below to various processes according to present invention.

1. Image Segmentation

In the invention, a feature region is defined as a region having richimage variations and higher brightness, and a non-feature region aredefined as others except the feature region. Generally, the gray scaleintensities of pixels in a non-feature region are relatively consistent.To extract different types of features from an image, the imagesegmentation according to present invention is based on gradientinformation and gray information. Gradient information is used toextract distinct boundaries from the image and gray-scale information isused to select brighter region. More preferably, variance informationmay be used in the image segmentation process, wherein variance may beused to extract boundaries of the image that are not distinct but needfurther enhancement (also referred to as “weak boundary”). In presentinvention, the segmentation of the image into a feature region and anon-feature region is the basis. The segmentation process will bedescribed in detail below.

Referring to FIG. 3, the system first shrinks the read ultrasound image,so as to increase the computation speed and decrease the impact from thenoise in the original image upon computation. For example, but notlimited to, the input image represented by its gray scale intensityI_(in), is shrunk to produce a shrunk image represented by its grayscale intensity I_(shrunk) whose side length is half as that of theoriginal. Then, the system segments the shrunk image I_(shrunk) by usingthe gradient information in the image, to produce a gradient imagetemplate. Next, the system uses the gray information of the imageI_(shrunk) to modify the gradient image template, to produce a grayimage template, and then the local variance information of the imageI_(shrunk) is used to modify the gray image template, to produce avariance image template. At last, a segmentation template is obtained bypost-processing the variance image template, wherein the feature regioncorrespond to the image region included in the segmentation template.Here, the gradient image template, the gray image template and thevariance image template have a relationship as follows: the gray imagetemplate includes image pixels within the gradient image template andthe variance image template includes image pixels within the gray imagetemplate. Detailed description will be made below to the gradientprocessing, gray-scale processing, variance processing and templatepost-processing thereof.

1.1 Gradient Processing

FIG. 4 shows the generation process of the gradient image template. Atstep S41, the gradient grad_(x) in the horizontal direction and thegradient grad_(y) in the vertical direction for each pixel in the shrunkimage I_(shrunk) are computed. To simplify computation, a simplifiedgradient G is defined as:G=max(|grad_(x)|,|grad_(y)|).

At step S42, a gradient threshold is so set that the number of pixels inthe image each having a gradient value greater than the gradientthreshold GTh accounts for a predetermined percentage (for example, 25%)of total pixels. At step S43, gradient G of each pixel is compared tothe threshold GTh; if G>GTh, a value in the gradient image templatecorresponding to said pixel is set to 1 in step 44; otherwise the valueset to 0 in step S45. The process is repeated until the gradient valuesof all the pixels in the image are compared. Finally, the resultedgradient image template shall include those pixels whose gradient valuesexceed the threshold GTh, and thus the distinct boundary information isextracted from the image.

1.2 Gray-Scale Processing

FIG. 5 shows the generation process of the gray image template. The grayscale intensity for each pixel in the shrunk image I_(shrunk) isdetermined, and then a gray threshold ITh is set, for example but notlimited herein, the threshold is set to multiples of the average grayscale intensity of the pixels included in the gradient image template(i.e., those pixels with a value of 1 in the gradient image template),such as 1.5. In this way, an initial gray image template is produced sothat it includes those pixels in the gradient image template and thepixels in the shrunk image I_(shrunk) of which gray scale intensitiesare greater than ITh (i.e., setting a value corresponding to a pixel inthe shrunk image I_(shrunk) with its gray scale intensity greater thanITh to 1 and then adding the corresponding value to the gradient imagetemplate), so as to extract the brighter region from the image. Toremove isolated small regions in the template, morphological filteringmay be effected on the initial gray image template. For example, theinitial gray image template is regarded as a binary image (a templatepoint thereon has a value of 1, and a non-template point thereon has avalue of 0), and then a 3*3 morphological erosion filtering and 5*5morphological dilation filtering is performed on the image, so as toproduce the final gray image template.

1.3 Variance Processing

FIG. 6 shows the generation process of the variance image template.First, at step S61, an appropriate smoothing is performed on the shrunkimage I_(shrunk), and then at steps S62, the variance Var is computedpixel-by-pixel as:

${Var} = {\sum\limits_{3*3{neighor}}\left\lbrack {I - {\underset{3*3{neighbor}}{E}(I)}} \right\rbrack^{2}}$

Where E represents an average over a neighbor domain of each pixel. Atstep S63, the global variance image is divided into blocks (for examplebut not limited to herein, each block having a size of 9*9), and themean variation of variance Block_Mean (Var) and Standard variation ofvariance Block_Std (Var) for each block are calculated. A local variancethreshold Block_VarTh is determined based on the mean variation and thestandard variation of variance.

For example (but not limited to), Block_VarTh=Block_Mean+Block_Std.

At steps S64 and S65, the variance Var is compared to the Block_VarThand the pixels each having a variation greater than Block_VarTh areadded into the gray image template. The process is repeated until allthe pixels are compared. Thus a final variance image template isproduced.

Here, the variance threshold Block_VarTh is a local threshold, so it'seasier to capture regions in which gradient values of pixels are notlarge in the whole but the local variations are relatively large,including the weak boundary. These regions usually include the portionshaving structural features in relatively uniform regions. Although thegradient values in the portions are not large, they tend to attract aviewer, and thus need enhancing particularly. It is noted that thevariance information needs to be used in combination with the gradientinformation; otherwise, for local regions having many strong boundaries,useful features may be lost when being segmented by only using thevariance information.

1.4 Template Post-Processing

FIG. 7 illustrates a diagram of a template post-processing in the imagesegmentation. First, small structures are removed from the aboveobtained variance image template, by using, for example (but not limitedto) 8-connectivity method. Then, a template modification and a templateframe-averaging are applied so as to produce the segmentation template.Wherein, the template frame-averaging process is very useful and itsdescription will be given below.

1.4.1 8-Connectivity

Removal of small structures with 8-connectivity method means to combineadjacent pixels into a connected region and remove the connected regionsin which the number of pixels is less than a predetermined thresholdvalue. Specifically, the 8-connectivity process comprises the followingsteps:

a) In the variance image template, searching for the template pixelsline by line, and the first searched pixel is marked as marker valueof 1. Hereafter, every time a template pixel is searched, the markervalue is incremented by 1. This process is repeated up to the end oftemplate. Thus a diagram of marker values is produced.

b) scanning the diagram of marker values, wherein for each pixel, itsmarker value is replaced by the minimum marker value in its 8neighboring pixels. The process is repeated until the marker values inthe diagram will no longer change. The pixels with the same marker valueform a connected region.

c) removing the connected region If the number of pixels in theconnected region is less than the predetermined threshold.

1.4.2. Template Modification

The template modification process is as follows:

1) rectifying the continuity of edges and structures of the image. Bysetting a region gradient-increasing threshold (for example, bymultiplying GTh with a constant less than 1, such as 0.85) and searchingaround the template pixels, if the gradient value of one non-templatepixel is greater than the threshold, the non-template pixel may be addedinto the template.

2) Further modifying the template to enable the template to have areasonable structure. For a template pixel, if the number of templatepixels in its 3*3 neighbors is below a constant (for example, 3), itwill be modified as a non-template pixel. For a non-template pixel, ifthe number of template pixels in its 3*3 neighbors is greater than aconstant (for example, 6), it will be modified as a template pixel. Theterm “template pixel” means a pixel with a value of 1 in the template,and “non-template pixel” means a pixel with a value of 0 in thetemplate.

1.4.3. Template Frame-Averaging

To reduce harmful effect caused by a sudden change of template betweenneighboring frames in the image sequence, the template post-processingin the image segmentation according to present invention includes aprocess of averaging the segmentation template of continuous frames,which comprises the following steps:

a) setting a template frame-averaging matrix M_(frame) _(—) _(ave), thevalue for each element in the matrix is initialized according to thetemplate modification result for the first image frame (or directlybased on the aforementioned variance image template or other templates,and the description thereof is omitted herein). For example, if anelement has a corresponding template point, the matrix value of theelement is set to maximum V_(max) (for example, 128+5), otherwise, ifthe element has no corresponding template point, the matrix value of theelement is set to minimum V_(min) (for example, 128-5). Here, as for theterm of “template point”, if a pixel in the segmentation template havingthe same location as in the template frame-averaging matrix is atemplate pixel, it is called as a corresponding template point, andcalled as a non-corresponding template point if it is a non-templatepixel.

b) for the second frame and subsequent frames, modifying the value foreach element in the matrix M_(frame) _(—) _(ave) according to thetemplate modification result. If an element has a corresponding templatepoint, its value is incremented by a constant (for example, 1). Thevalue of an element having a non-corresponding template points will bedecreased by a constant (for example, 1). If the value of an element isgreater than V_(max), its value is set to V_(max); if the value for anelement is below V_(min), its value is set to V_(min).

c) determining the segmentation template for the current frame accordingto the matrix M_(frame) _(—) _(ave). For example, the segmentationtemplate may be set to only include the pixels corresponding to elementsin the matrix each having a value greater than the predetermined value(for example, 128).

In this way, for a feature region newly added to the image template ofthe current frame, the feature region may be filled in as a part of thesegmentation template only if several continuous frames for the featureregion satisfy a condition of segmentation into the template (e.g. thepixel value of the corresponding point in the template frame-averagingmatrix values is greater than 128). Thereby, the possibility of suddenchange of template between frames is effectively reduced. As shown inFIG. 8, after the image segmentation blocks (excluding the templateframe-averaging block) segment a new portion in the N+1^(st) frame, thetemplate doesn't expand immediately, but updates the corresponding partin M_(frame) _(—) _(ave). If this portion is to be segmented into thetemplate in the subsequent frames, it will become a part of the templateat the N+m^(th) frame.

Once the segmentation template is determined, the feature region andnon-feature region are determined. For example, the regions with valueof 1 in the segmentation template are set as a feature region and theregions with value of 0 are set as a non-feature region. As shown inFIG. 17, the system will adopt different data processes on the featureregions and the non-feature regions, respectively. FIG. 17 shows apreferred embodiment, where anisotropy smoothing and anisotropysharpening are performed on the image data classified as a featureregion pixel-by-pixel, then the sharpened feature regions are mergedwith the input image I_(in), so as to produce the enhanced result of thefeature region. Local gray-scale adjustment is performed on the imagedata classified as a non-feature region so as to smooth the image forthe purpose of mitigation of speckle noise, and then a detection andremoval process of black speckles is further performed, thereby thenon-feature region is enhanced. The above implementation mode, however,does not limit the scope of present invention, and equivalent variationsalso fall within the scope according to present invention. For example,the merging of the sharpened data in the feature regions and the inputimage I_(in), may be delayed until the feature regions are to be mergedwith the non-feature regions.

2. Data Processing on Feature Regions

2.1 Anisotropy Smoothing

FIG. 9 shows a process of anisotropy smoothing the image data classifiedas the feature region. In a feature region (may correspond to the shrunkimage I_(shrink) of the input image or directly correspond to the inputimage I_(in)), several pixels centered on each pixel (as shown in FIG.10, for example, but not limited to taking three pixels) are selected inthe angle direction of 0°, 45°, 90° and 135° to compute theorientational variance in each direction, and the direction in which theorientational variance is at maximum is set as the orientation of thispixel. A small neighbor domain is selected around each pixel (forexample, a 5*5 neighbor domain), centered on this pixel and based on theorientation of the pixel, the number of pixels in each direction in theneighbor domain of a pixel is counted and then the direction in whichthe number of pixels is at the maximum is determined as the localdominant orientation of the pixel. The local dominant orientation may beunderstood as a smoothed local gradient direction. Centered at thepixel, in a direction orthogonal to the local dominant orientation, aneighbor domain is selected to perform data smoothing on the pixel (forexample, when the local dominant orientation is a zero degree directionand its orthogonal direction is a 90 degree direction, the neighbordomain includes the pixel and several pixels above and below it). Forexample, a weighted average of various pixels in the neighborhood may becomputed, as the smoothed result represented by its gray scale intensityI_(smoothed).

2.2 Anisotropy Sharpening

FIG. 11 shows the anisotropy sharpening process. An orientationalLaplacian may be computed for the smoothed result I_(smoothed) accordingto the local dominant orientation. For example, if the local dominantorientation for a pixel is the horizontal direction, three pixels, I⁻¹,I₀ and I₊₁ are taken in the horizontal direction centered at the pixelI₀, where I represents the gray-scale Intensity of the pixel, and0⁰Laplacian=−I ⁻¹+2*/I ₀ +I ₊₁

is calculated. By multiplying the orientational Laplacian with animage-sharpening coefficient R_(sharpen), the anisotropically sharpenedresult I_(sharpened) is generated. The calculation of R_(sharpen) isgiven by:R _(sharpen) =C*std*(M _(frame) _(—) _(ave) −MID)/5,

Where std is the standard variation for the gray image in theneighborhood (for example, a 3*3 neighbor) of each pixel in the shrunkimage I_(shrunk) (but not limited to I_(shrunk), the input image orvarious smoothed or distorted image of the input image is alsopossible), std=√{square root over (E[I−E(I)]²)}

M_(frame) _(—) _(ave) is the template frame-averaging matrix obtained inimage segmentation, MID is a predetermined value corresponding to thetemplate frame-averaging matrix. In this embodiment, MID=128, C is apredetermined constant, for example, but not limited to C=20, and itsvalue may be selected according to different ultrasound systems. Thesharpening coefficient derived from the standard variation std presentsthree advantages. First, boundaries may be highlighted very nicely,because the standard variation in regions including edges of image isrelatively larger. Second, the standard variation is smaller at theboundary of the template (i.e. the segmenting lines between the featureregions and non-feature regions), and the enhancement effect isrelatively weak here, thus the transition from feature regions tonon-feature regions is perfectly resolved without introducing artificialboundary. Third, the reliance of the enhancement effect upon thesegmentation template is reduced significantly. Assuming thesegmentation template includes excessively uniform regions, thedisadvantageous effect caused by mistaken enhancement may be ignoredbecause the standard variations in these uniform regions are very small.Here, the sharpening coefficient R_(sharp) relates to the matrixM_(frame) _(—) _(ave), so that the image presents a progressivetransition process in time when the segmentation template changes, toavoid occurrence of image flare.

2.3 Image Fusion

FIG. 12 shows the merging of the orientational sharpened result of thefeature region and the input image. The orientational sharpened resultis enlarged up to the size of the input image (by using bilinearinterpolation or other interpolation) and then multiplied with anenhancement coefficient selected by the user (e.g. a coefficient valuedin a range of 1-2), and sequentially added to the input image I_(in), toproduce an enhanced result of the feature region.

3. Data Processing on Non-Feature Region

3.1 Local Gray-Scale Adjustment

FIG. 13 shows a process of local gray-scale adjustment of thenon-feature region according to a preferred embodiment, in which a localgray-scale compression algorithm is applied to, e.g., the input imageI_(in) to smooth the image. First, the image I_(in) is divided intoblocks (for example, but not limited to, blocks of 11*11), wherein someblocks may be overlapped with others at least in part to mitigateblocking effect, and a gray-scale mean Mean_(block) is computed for eachof the blocked images, to produce a mean images. The gradient Gcalculated in the image segmentation is scaled up to the size of theinput image. For a relatively large G (for example, G is greater than10), the gray-scale adjustment coefficient R_(adjust) is set to arelatively large value (for example, 0.75), and for a relatively smallG, R_(adjust) is set to a relatively small value (for example, 0.6).After local gray-scale adjustment, an adjusted gray-scale intensityI_(adjusted) is computed with the following equation:I _(adjusted)=Mean_(block)+(I _(in)−Mean_(block))*R _(adjust).

After the local gray-scale adjustment, the local gray-scale for theimage will decrease, which may be interpreted as the gray-scalehistogram in local regions is compressed (as shown in FIG. 14, beingcompressed at the center of the local average gray scale) and it presentto the eyes a sense of the image being smoothed. At the same time, forpixels with larger gradients, R_(adjust) becomes larger accordingly andthus local fine features may be kept.

3.2 Black Speckle Mitigation

After a local gray-scale adjustment, a process of black speckledetection and mitigation may be further applied, as shown in FIG. 15.The preferred embodiment provides a method for detecting andcompensating the black speckles, first, for the local gray-scaleadjusted image I_(adjusted), its image gray mean I_(mean) is computed bymeans of a neighbor domain of each pixel with variable size. Then,I_(mean) is subtracted from the adjusted image I_(adjusted), to producea gray difference I_(diff). A pixel with the corresponding differenceI_(diff) greater than a predetermined value (depending on differenttypes of ultrasound systems) is determined as a compensating pixel, andthe corresponding difference I_(diff) of the compensating pixel ismultiplied with a constant, to produce a compensating image representedby its intensity I_(modify). A value corresponding to a non-compensatingpixel in the compensating image is set to 0 (in another word, thenon-compensating pixels are compensating pixels with compensating valueof 0). At last, I_(modify) may be added to I_(adjusted) directly or viaproper averaging, to remove speckles from the non-feature regions.

As best shown in FIG. 16, “a” represents a black speckle with itsgray-scale below the surrounding pixels, and its gray scale intensitywill change significantly after averaging over the surrounding smallneighbors; “b” is within a large structure with consistent gray scaleintensities and the gray scale intensities vary little after averaging;and “c” is on the edge of a large structure and the variation in grayscale intensities is medial after averaging. A proper threshold can beselected to distinguish the cases of “a” and “c”, so as to detect thespeckles. The speckles have very different sizes in near field and farfield of the image, so neighbors with different sizes are needed toproduce a mean image I_(mean), and the size of the neighbor domainshould be suitable for the sizes of speckles to be removed (for example,for an image with size 500*400, the size of neighbor domain in the nearfield may be selected to be 5*5, the size in the medium field to be 7*7and the size in the far field to be 9*9). In this way, black speckleswith a predetermined size (determined by the size of neighbors to beaveraged) may be detected and the compensation amount may be determinedaccording to the gray-scale difference between the speckles and thesurrounding pixels. By such a processing, black speckles may bemitigated effectively while bringing very small blurring effect to otherregions.

Please note that, the above local gray level adjustment is not necessaryfor the mitigation of black speckle, said mitigation may be applieddirectly to the input image I_(in) without local gray scale adjustment,and the procedure is similar.

The above embodiments had been tested by a (but not limited herein)black/white digital B-mode ultrasonic system, which verifies that theimage quality may be improved greatly by using the method of imageenhancement and speckle mitigation of present invention.

While the above descriptions are made to some specific embodiments, it'sto be noted that these embodiments are merely illustrative and preferredand that a person skilled in the art may make various changes andmodifications to the embodiments. As can be seen from the aboveembodiments, the segmentation template is obtained through a templateconstruction process based on gradient, gray-scale and variance, andtemplate post-processing based on small structure removal, templatemodification and especially template frame-averaging, and thus the imageis segmented into a feature region and a non-feature region based on thesegmentation template, Wherein the template construction process is abasis for image enhancement, so a person skilled in the art can producea segmentation template with existing technology in the prior arts basedon the constructed template of present invention, including (or notincluding) template post-processing to modify the template. Comparedwith prior arts, the resulted ultrasound image is also enhancedcorrespondingly.

Furthermore, even in the above template construction process, thevariance processing therein is not necessary, for example. Althoughvariance processing may reflect the true structure information finely,it may be omitted and only gradient and gray-scale processing is used incase image quality is not expected to be very high, which may simplifycomputation and save cost.

Similarly, in the template post-processing according to presentinvention, not all of the 8-connectivity, template modification andtemplate frame-averaging provided in the invention are necessary. Forexample, it is possible to apply the template frame-averaging to thevariance template, which has been pointed out in the previousembodiments.

Moreover, in the embodiments according to present invention, the dataprocessing on the feature regions is exemplary. For example, in thepractice of present invention, it is also possible to only chooseanisotropy smoothing based on the specific situation, to simplifycomputation and processing.

It is possible to make various combinations and deletions of thedisclosed embodiments and therefore, the scope according to presentinvention is to be defined by the appended claims.

1. A method for enhancing an ultrasound image for use in an ultrasoundimaging system, comprising: using the processor to perform the steps of:reading the ultrasound image; segmenting the ultrasound image into afeature region and a non-feature region based on gradient informationand gray-scale information in the image, and then performing differentdata processing on the image classified as the feature region and thenon-feature region respectively; merging the processed feature regionand non-feature region to produce an enhanced Image; wherein the step ofsegmenting the image comprises a step of extracting distinct boundariesin the ultrasound image by means of the gradient information and a stepof extracting brighter regions in the ultrasound image by means of thegray-scale information to generate a gray image template; shrinking theread ultrasound image, wherein the step of extracting the distinctboundaries comprises sub-steps of: setting a gradient threshold suchthat the number of pixels in the shrunk image each having a gradientvalue greater than the gradient threshold accounts for a predeterminedpercentage of total pixels; and generating a gradient image template sothat the gradient image template includes the pixels each having agradient value greater than the gradient threshold; and wherein the stepof extracting the brighter regions comprises sub-steps of: setting agray threshold; and generating an initial gray image template so thatthe initial gray image template includes the pixels in the gradientimage template and pixels in the shrunk image each having a gray scaleintensity greater than the gray threshold.
 2. The method according toclaim 1, wherein the gray threshold is set to multiples of an averagegray scale intensity of the pixels included in the gradient imagetemplate.
 3. The method according to claim 1, wherein the step ofsegmenting the image further comprises a step of extracting weakboundaries in the image by means of variance information in theultrasonic image, to generate a variance image template.
 4. The methodaccording to claim 3, wherein the step of extracting the weak boundariescomprises sub-steps of: computing a variance of each pixel in the image,to produce a global variance image; dividing the global variance imageinto blocks and computing a local variance threshold of each block; andgenerating the variance image template by adding into the gray imagetemplate the pixels each having a variance greater than the localvariance threshold.
 5. The method according to claim 4, wherein the stepof segmenting the image further comprises: a step of post-processing thevariance image template to produce a segmentation template, wherein thefeature region being set as the image region indicated by thesegmentation template.
 6. The method according to claim 5, wherein thestep of post-processing comprises a step of removing small structures inthe image template.
 7. The method according to claim 6, wherein the stepof post-processing further comprises a step of rectifying continuity ofedges and structures of the image template.
 8. The method according toclaim 7, wherein the step of post-processing further comprises a step ofaveraging the segmentation template of continuous frames, wherein for afeature region newly added to the image template of current frame, thefeature region is filled in as a part of the segmentation template onlyif several continuous frames for the feature region satisfy asegmentation condition.
 9. The method according to claim 8, wherein thestep of averaging the segmentation template further comprises sub-stepsof: a) setting a template frame-averaging matrix M_(frame) _(—) _(ave),the value for each element in the matrix being initialized according tothe image template for a first frame; b) modifying the value for eachelement in the matrix according to the image template for a second frameand subsequent frames; and c) determining the segmentation template forthe image of the current frame according to the modified matrixM_(frame) _(—) _(ave).
 10. The method according to claim 9, the dataprocessing on the image data classified as the feature region comprisesthe steps of: (1) anisotropy smoothing the image data, comprisingsub-steps of: determining a local dominant orientation for each pixel;and smoothing each pixel in a direction normal to the local dominantorientation; and (2) anisotropy sharpening the smoothed pixels,comprising sub-steps of: computing an orientational Laplacian for eachof the smoothed pixels in the local dominant orientation; andmultiplying the orientational Laplacian with an image-sharpeningcoefficient R_(sharpen), to generate an anisotropically sharpenedresult.
 11. The method according to claim 10, wherein the imagesharpening coefficientR _(sharpen) =C*std*(M _(frame) _(—) _(ave) −MID)/5, where C is apredetermined constant, std is the standard variation for gray imagewithin a neighborhood of each pixel, std=√{square root over(E[I−E(I)]²)}, wherein I represents the gray scale intensity of a pixel,M_(frame) _(—) _(ave) is the template frame-averaging matrix used whensegmenting the image, and MID is a predetermined value corresponding tothe template frame-averaging matrix.
 12. The method according to claim11, further comprising steps of: enlarging the anisotropically sharpenedresult up to the size of the input ultrasonic image; multiplying theenlarged result with a selectable enhancement coefficient and thenadding the multiplied result to the input ultrasonic image, so as toproduce a image with the feature region enhanced.
 13. The methodaccording to claim 1, wherein the data processing on the image dataclassified as the non-feature region comprising a step of smoothing theimage by means of a local gray-scale adjustment of various pixels,comprising sub-steps of: a) dividing the input image into blocks, andcomputing a gray scale mean of each blocked image, Mean_(block); b)setting different gray-scale adjustment coefficients R_(adjust), basedon the different gradient values of various pixels in the inputultrasonic image; and c) computing an adjusted gray scale intensityI_(adjusted) to produce a local gray-scale adjusted image,I_(adjusted)=Mean_(block)+(I_(in)−Mean_(block))*R_(adjust), where I_(in)representing the gray scale intensity of the input image.
 14. The methodaccording to claim 13, wherein the data processing on image dataclassified as the non-feature region further comprises a step ofdetecting and compensating black speckles which comprises sub-steps of:i) computing a mean image represented by its gray scale intensityI_(mean) for the input image I_(in) or the local gray-scale adjustedimage I_(adjusted) by using a neighbor domain of each pixel withvariable size; ii) determining a compensating pixel and its respectivecompensating amount based on the mean image I_(mean); and iii) addingthe compensating amount for each compensating pixel into the input imageI_(in) or the local gray-scale adjusted image I_(adjusted), to removethe speckles from the non-feature regions.
 15. The method according toclaim 14, wherein the compensating pixel means a pixel for which animage difference I_(diff) obtained by subtracting the mean imageI_(mean) from the input image I_(in) or the local gray-scale adjustedimage I_(adjusted) is greater than a predetermined value; and thecompensating amount for the compensating pixel is multiples of therespective difference I_(diff).
 16. The method according to claim 11,wherein the data processing on image data classified as the non-featureregion comprising a step of smoothing the image by means of a localgray-scale adjustment of various pixels, comprising sub-steps of: a)dividing the input image into blocks, and computing a gray mean of eachblocked image, Mean_(block); b) setting different gray-scale adjustmentcoefficients R_(adjust) based on the different gradient values ofvarious pixels in the input image; and c) computing an adjusted grayscale intensity I_(adjusted) to produce a local gray-scale adjustedimage, I_(adjusted)=Mean_(block)+(I_(in)−Mean_(block))*R_(adjust), whereI_(in) represents a gray scale intensity.
 17. The method according toclaim 16, wherein the data processing on image data classified as thenon-feature region further comprises a step of detecting andcompensating black speckles, which comprise sub-steps of: i) computing amean image represented its gray scale intensity I_(mean) for the inputimage I_(in) or the local gray-scale adjusted image I_(adjusted) byusing a neighbor domain of each pixel with variable size; ii)determining a compensating pixel and its respective compensating amountbased on the mean image I_(mean); and iii) adding the compensatingamount for each compensating pixel into the input image I_(in), or thelocal gray-scale adjusted image I_(adjusted), to remove speckles fromthe non-feature regions.
 18. The method according to claim 17, whereinthe compensating pixel means a pixel for which a image differenceI_(diff) obtained by subtracting the mean image I_(mean) from the inputimage I_(in) or the local gray-scale adjusted image I_(adjusted) isgreater than a predetermined value; and the compensating amount for thecompensating pixel is multiples of the respective difference I_(diff).19. A method for mitigating speckles in an input ultrasonic image bylocally adjusting a gray scale intensity of the image classified as anon-feature region, comprising using the processor to perform thesub-steps of: a) dividing the input image into blocks, and computing agray mean of each blocked image, Mean block; b) setting differentgray-scale adjustment coefficients R adjust, based on the differentgradient values of various pixels in the input image; and c) computingan adjusted gray scale intensity I adjusted to produce a localgray-scale adjusted imageIadjusted=Mean block+(Iin−Mean block)*Radjust, where Iin represents thegray scale intensity of the input image; detecting and compensatingblack speckled which includes the steps of: d) computing a mean imagerepresented by its gray scale intensity I mean for the local gray-scaleadjusted image I adjusted by using a neighbor domain of each pixelthereof with variable size; e) determining a compensating pixel and itsrespective compensating amount based on the mean image I mean; and f)adding the compensating amount for each compensating pixel into thelocal gray-scale adjusted image I adjusted, to remove the speckles fromthe non-feature regions; the compensating amount for the respectivecompensating pixel is multiples of the respective difference I diff.